The present invention relates to a method and a device of light source discrimination, skin color correction, and color image correction, and storage medium thereof capable of being read by a computer. More to particularly, this invention relates to a method and a device of light source discrimination, skin color correction, and color image correction, and storage medium thereof capable of being read by a computer which are intended to correct color in a color image based on skin area in a color image.
When there is displayed color image inputted by a color image input device such as video camera, digital camera, and so forth to a computer screen and so forth, in many cases, color reproduction is insufficient (color representation gives a sense of incompatibility to persons). In particular, when a face of person is included in image, requirement in terms of accuracy of color reproduction is severe. By way of problem often occurs, in some cases, image is reproduced in a different color from original color because there is taken photograph under different light source such as a fluorescent light from daylight which is original light for this photograph. Formerly, there is known a computer software in order to correct such image to a natural color representation, that is known by way of a photo retouch software.
In such the software, for instance, there is an image whose green component is exceedingly increased because it is taken photograph under the fluorescent light. Concerning this image, there is implemented a conversion based on curved line as shown in FIG. 1, toward respective three primary color components of red (R), green (G), and blue (B) for the color image, thus an operator corrects color in accordance with his experience, while suppressing green component, and increasing red component. Consequently, RGB value (horizontal axis) of input is converted into Rxe2x80x2Gxe2x80x2Bxe2x80x2 value (vertical axis) of output. Further, there is known a software enabling such empirical correction processing to be simply realized in certain degree by only an amateur.
However, the conventional image processing (color correction) implemented as described above is known content as it is by way of know-how. The operator should specify content of processing. The content of processing was determined in accordance with the know-how of the operator or the sense of the operator. There is the problem that reproducibility of color is low, because it is incapable of being unnecessary to be executed the processing with appropriate content. Concretely, there are following matters depending on those problems.
Firstly, there can not be implemented appropriate color correction in terms of the image given previously, because processing itself (operation) depends on sensitive ability of person.
Secondly, since processing itself (operation) depends on sensitive ability of person, different result occurs in every operator.
Lastly, for instance, when there is used the software in order to correct green excessive image, the result of correction is different in every operator, because the operator judges whether or not the image is green excessive image, in accordance with his sense about how much degree, and inputs the result thereof to the computer as an information.
In view of the foregoing, it is an object of the present invention, in order to overcome the above mentioned problems, to provide a method and a device of light source discrimination, skin color correction, and color image correction, and a storage medium capable of being read by a computer which enable reproducibility of color to be maintained always highly regardless of difference of light source.
There will be described the principle of color correction in the present invention. There is taken spectral reflectance (reflectance in every each wavelength xcex) of skin or a face to be xcex1(xcex). Spectral radiant intensity of a light source is taken to be S(xcex). Spectral sensitivity of three kinds of sensor of color image input device are taken to be r(xcex), g(xcex), b(xcex). At this time, there is represented outputs R, G, B, of respective sensors by formula 1.                     {                                                            R                =                                                      ∫                    380                    780                                    ⁢                                                            r                      ⁡                                              (                        λ                        )                                                              ⁢                                          β                      ⁡                                              (                        λ                        )                                                              ⁢                                          S                      ⁡                                              (                        λ                        )                                                              ⁢                                          xe2x80x83                                        ⁢                                          ⅆ                      λ                                                                                                                                              G                =                                                      ∫                    380                    780                                    ⁢                                                            g                      ⁡                                              (                        λ                        )                                                              ⁢                                          β                      ⁡                                              (                        λ                        )                                                              ⁢                                          S                      ⁡                                              (                        λ                        )                                                              ⁢                                          xe2x80x83                                        ⁢                                          ⅆ                      λ                                                                                                                                              B                =                                                      ∫                    380                    780                                    ⁢                                                            b                      ⁡                                              (                        λ                        )                                                              ⁢                                          β                      ⁡                                              (                        λ                        )                                                              ⁢                                          S                      ⁡                                              (                        λ                        )                                                              ⁢                                          xe2x80x83                                        ⁢                                          ⅆ                      λ                                                                                                                              [FORMULA  1]            
Here, wavelength xcex is a value with nm as a unit.
Radiant intensity of daylight is taken to be S1(xcex), at this time, outputs of respective sensors are (R1, G1, B1). Radiant intensity of a fluorescent light is taken to be S2(xcex), at this time, outputs of respective sensors are (R2, G2, B2). Difference between these sensor outputs brings unnatural skin color. Generally, it is incapable of being discriminated whether radiant intensity of light source is S1(xcex) or S2(xcex) only seeing an image. However, skin is taken photograph in an image, and reflectance xcex1(xcex) is known. When the sensor output (R, G, B) equals to (R1, G1, B1), it is capable of being known that a light source is daylight. Similarly, when the sensor output (R, G, B) equals to (R2, G2, B2), it is capable of being known that a light source is a fluorescent light. Practically, reflectance xcex1(xcex) of skin which is different according to a person, is not constant. It is difficult that sensor output agrees therewith completely as above. However, it is possible to estimate a light source whose possibility is high, by checking distribution of sensor output in terms of skin color of respective light sources.
According to a first aspect of the present invention, in order to achieve the above-mentioned object, there is provided a light source discrimination method which comprises the steps of calculating color distribution of skin area of a color image, comparing the color distribution calculated previously in the above calculating step with a color distribution already known of skin taken photograph under a plurality of different lighting light sources, and discriminating a lighting light source under which the color image is taken photograph, based on result of the comparison.
According to a second aspect of the present invention, there is provided a light source discrimination device which comprises a color distribution calculation means for calculating color distribution of skin area of a color image, a reference color distribution storage means for storing therein color distribution of skin taken photograph under a plurality of different lighting light sources, and a light source discrimination means for discriminating lighting light source under which the color image is taken photograph, while comparing color distribution of skin area of the color image calculated by the color distribution calculation means with color distribution of skin under a plurality of lighting lights, which is stored in the reference color distribution storage means.
According to a third aspect of the present invention, there is provided a skin color correction method which comprises the steps of calculating color distribution of skin area of a color image, comparing color distribution calculated previously at the above calculating step with color distribution already known of skin taken photograph under a plurality of different lighting light sources, and correcting color of the skin area into a skin color under prescribed lighting light source based on result of the comparison.
According to a fourth aspect of the present invention, in the third aspect, there is provided a skin color correction method, wherein correction of color of the skin area is implemented based on result of discrimination while discriminating lighting light source under which the color image is taken photograph.
According to a fifth aspect of the present invention, in the third or fourth aspect, there is provided a skin color correction method, wherein it causes the skin area of the color image to be detected automatically.
According to a sixth aspect of the present invention, there is provided a color image correction method which comprises the steps of calculating color distribution of skin area of a color image, comparing the color distribution calculated previously at the calculating step with color distribution already known of skin taken photograph under a plurality of different lighting light sources, and correcting color of pixel in the color image into color under prescribed lighting light source based on the result of comparison.
According to a seventh aspect of the present invention, in the sixth aspect, there is provided a color image correction method, wherein correction of pixel in the color image is implemented based on result of discrimination, while discriminating lighting light source under which the color image is taken photograph.
According to an eighth aspect of the present invention, in the sixth or seventh aspect, there is provided a color image correction method, wherein it causes skin area of the color image to be detected automatically.
According to ninth aspect of the present invention, there is provided a skin color correction device which comprises a light source discrimination means for discriminating lighting light source at the time when the color image is taken photograph on the basis of color distribution of skin area of color image, a skin color storage means for storing therein skin color corresponding to a plurality of lighting light sources when skin is taken photograph under a plurality of different lighting light sources, and a skin color conversion means for converting a skin color of the skin area into a skin color read out from the skin color storage means, while reading out skin color under prescribed lighting light source corresponding to a skin color of the skin area, from the skin color storage means on the basis of result of discrimination of the light source discrimination means and a skin color of the skin area.
According to a tenth aspect of the present invention, there is provided a skin color correction device which comprises a light source discrimination means for discriminating lighting light source when a color image is taken photograph on the basis of color distribution of skin area of the color image a skin color output means for outputting a skin color under prescribed lighting light source of a skin color, while inputting result of discrimination of the light source discrimination means and the skin color of the skin area, and a skin color conversion means for converting a skin color of the skin area into a skin color outputted from the skin color output means.
According to an eleventh aspect of the present invention, there is provided a skin color correction device which comprises a light source discrimination means for discriminating lighting light source when a color image is taken photograph on the basis of color distribution of skin area of the color image a skin color calculation means for calculating skin color under prescribed lighting light source of the skin color on the basis of result of discrimination of the light source discrimination means and a skin color of the skin area, and a skin color conversion means for converting a skin color of the skin area into a skin color calculated by the skin color calculation means.
According to a twelfth aspect of the present invention, in any one of the ninth to eleventh aspect, there is provided a skin color correction device, further comprising a skin area detection means for detecting a skin area automatically from the color image.
According to a thirteenth aspect of the present invention, there is provided a color image correction device which comprises a light source discrimination device for discriminating lighting light source when a color image is taken photograph on the basis of color distribution of a skin area of the color image, a reference color storage means for storing therein color corresponding to a plurality of lighting light sources when a substance is taken photograph under a plurality of different lighting light sources, and a color conversion means for converting a color of pixel into a color read out from the reference color storage means, while reading out color under prescribed lighting light source corresponding to a color of the pixel, from the reference color storage means on the basis of result of discrimination of the light source discrimination means and a color of the pixel in the color image.
According to a fourteenth aspect of the present invention, there is provided a color image correction device which comprises a light source discrimination means for discriminating lighting light source when a color image is taken photograph on the basis of color distribution of skin area of the color image, a color output means for outputting color under prescribed lighting light source of the color, while inputting result of discrimination of the light source discrimination means and a color of pixel of the color image, and a color conversion means for converting a color of the pixel into a color outputted by the color output means.
According to a fifteenth aspect of the present invention, there is provided a color image correction device which comprises a light source discrimination means for discriminating lighting light source when a color image is taken photograph on the basis of color distribution of skin area of the color image, a color calculation means for calculating a color under prescribed lighting light source of the color on the basis of result of discrimination of the light source discrimination means and a color of pixel in the color image, and a color conversion means for converting a color of the pixel into a color calculated by the color calculation means.
According to a sixteenth aspect of the present invention, in any one of aspects 13 to 15, there is provided a color image correction device, further comprising a skin area detection means for detecting a skin area automatically from the color image.
According to a seventeenth aspect of the present invention, there is provided a storage medium which is capable of being read by a computer, storing therein a program for realizing a means for calculating color distribution of skin area of color image, a means for obtaining color distribution of skin taken photograph under a plurality of different lighting light source, and a means for discriminating lighting light source under which the color image is taken photograph while comparing a color distribution of a skin area of the color image calculated by the means for calculating with a color distribution of skin under a plurality of lighting light sources obtained by the means for obtaining.
According to an eighteenth aspect of the present invention, there is provided a storage medium which is capable of being read by a computer, storing therein a program for realizing a means for discriminating lighting light source when a color image is taken photograph on the basis of a color distribution of a skin area of the color image a means for obtaining skin colors corresponding to a plurality of different lighting light sources when skin is taken photograph under a plurality of different lighting light sources a means for converting a skin color of a skin area into a skin color received from the means for obtaining, while receiving a skin color under prescribed lighting light source corresponding to a skin color of the skin area from the means for obtaining on the basis of result of discrimination by the means for discriminating and a skin color of the skin area.
According to a nineteenth aspect of the present invention, there is provided a storage medium which is capable of being read by a computer, storing therein a program for realizing a means for discriminating lighting light source when a color image is taken photograph on the basis of a color distribution of a skin area of the color image, a means for outputting a skin color under prescribed lighting light source of a skin color, while receiving the result of discrimination by the means for discriminating and the skin color of the skin area, and a means for converting a skin color of the skin area into a skin color outputted by the means for outputting.
According to a twentieth aspect of the present invention, there is provided a storage medium which is capable of being read by a computer, storing therein a program for realizing a means for discriminating lighting light source when a color image is taken photograph on the basis of a color distribution of a skin area of the color image, a means for calculating a skin color under prescribed lighting light source of a skin color on the basis of result of discrimination by the means for discriminating and the skin color of the skin area, and a means for converting a skin color of the skin area into a skin color calculated by the means for calculating.
According to a twenty-first aspect of the present invention, in any one of aspects 18 to 20, there is provided a storage medium which is capable of being read by a computer, storing therein a further program for realizing a means for detecting a skin area automatically from the color image.
According to a twenty-second aspect of the present invention, there is provided a storage medium which is capable of being read by a computer, storing therein a program for realizing a means for discriminating lighting light source when a color image is taken photograph on the basis of a color distribution of a skin area of the color image a means for obtaining colors corresponding to a plurality of different lighting light sources when a substance is taken photograph under a plurality of different lighting light sources a means for converting a color of pixel into a color received from the means for obtaining, while receiving a color under prescribed lighting light source corresponding to a color of the pixel from the means for obtaining on the basis of result of discrimination by the means for discriminating and a color of said pixel in the color image.
According to a twenty-third aspect of the present invention, there is provided a storage medium which is capable of being read by a computer, storing therein a program for realizing a means for discriminating lighting light source when a color image is taken photograph on the basis of a color distribution of a skin area of the color image a means for outputting a color under prescribed lighting light source of a color, while receiving the result of discrimination by said means for discriminating and the color of pixel in the color image; and a means for converting a color of the pixel into a, color outputted by the means for outputting.
According to a twenty-fourth aspect of the present invention, there is provided a storage medium which is capable of being read by a computer, storing therein a program for realizing a means for discriminating lighting light source when a color image is taken photograph on the basis of a color distribution of a skin area of the color image a means for calculating a color under prescribed lighting light source of a color on the basis of result of discrimination by the means for discriminating and the color of pixel in the color image, and a means for converting a color of pixel into a color calculated by the means for calculating.
According to a twenty-fifth aspect of the present invention, in any one of aspects 22 to 24, there is provided a storage medium which is capable of being read by a computer storing therein a further program for realizing a means for detecting a skin area automatically from the color image.